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^Thls  report  presents  a  theory  of  the  optimal  display  format  for 
tasks  that  have  multiple  stimulus  elements.  Our  previous  research 
indicates  that  these  various  elements  should  be  presented  to  display 
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the  various  display  elements  are  correl ated  (e.g.,  temperature  and 
pressure  of  gas  in  a  pipe),  (b)  the  separate  elements  must  be 
Integrated  into  a  single  mental  model  of  the  environment. 
Collectively,  we  define  these  two  conditions  as  the  degree  of 
correlation/integration.  As  correlation/  integration  increases,  the 
relative  advantages  of  separate  resources  decreases.  The  research  in 
our  own  and  other  laboratories  that  supports  this  concept  is  reviewed 
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The  Limits  of  Multiple  Resource  Theory: 

The  Role  of  Task  Correlation/ Integration  In  Optimal  Display  Formatting 
Christopher  D.  Wickens  &  David  B.  Boles 

The  purpose  of  this  report  is  to  describe  a  theoretical  framework 
for  predicting  the  limits  of  application  of  multiple  resource  theory  to 
dual  task  performance.  In  brief,  this  theory  proposes  that  two  tasks 
will  be  time-shared  less  efficiently  if  they  impose  demands  on  the 
common  processing  resources  shown  in  Figure  1  (Wickens,  in  press, 

1984).  The  particular  theoretical  framework  is  meant  to  resolve  some 
seeming  contradictions  that  exist  in  the  experimental  data  relating 
task  similarity  to  dual  task  time-sharing  efficiency.  On  the  one  hand, 
we  have  demonstrated  in  several  experiments  that  the  more  similar  two 
tasks  are,  in  terms  of  their  competition  for  processing  resources 
(l.e.,  input  and  output  modalities,  central  processing  codes),  in  the 
multiple  resources  model,  the  greater  will  be  the  degree  of  task 
interference  (e.g.,  Wickens,  Sandry,  &  Vidulich,  1983;  Vidullch  & 
Wickens,  1982).  On  the  other  hand,  there  are  a  more  diverse  set  of 
results  coming  from  a  variety  of  sources  that  suggest  the  opposite 
conclusion:  Time-sharing  efficiency  Improves  with  greater  similarity 
between  tasks  and  task  elements.  In  the  following  pages  we  shall 
describe  these  different  classes  of  results,  and  then  attempt  to 
Integrate  the  two  contradictory  classes  of  results  within  the  framework 
of  a  testable  theory. 

1.  Attention  switching.  LaBerge,  VanGelder,  and  Yellott  (1971) 
compared  the  time  it  took  to  switch  attention  to  unexpected  stimuli 
that  were  presented  along  the  auditory  or  visual  modality.  The  stimuli 
were  lights  or  tones  that  required  a  left  or  right  keypress  response. 
Attention  was  aligned  to  an  expected  sensory  channel  by  a  probablistlc 
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cueing  technique,  and  the  experimenters  examined  the  delay  In  RT  that 
resulted  when  a  stimulus  occurred  on  a  non-attended  channel  (I.e.,  in 
the  stimulus  modality  that  was  not^  expected  on  a  particular  trial). 
LaBerge  et  al .  found  an  approximate  100  msec  increase  when  attention 
needed  to  be  switched  between  modalities,  in  contrast  with  conditions 
when  attention  could  be  maintained  within  a  modality.  In  these  results 
there  is  a  cost  to  cross -modality  stimulation  that  appears  to  run 
contrary  to  the  view  of  multiple  resource  theory  which  predicts  that 
cross-modal  time-sharing  will  be  uperior  to  intra-modal. 

These  results  have  been  echoed  to  some  extent  by  other  research  in 
our  laboratory,  where  the  advantages  of  employing  separate  input 
modalities  have  not  always  been  terribly  robust  (i.e.,  Vidulich  & 
Wickens,  1982),  and  have  sometimes  been  absent  altogether  (Tsang,  1983; 
Uickens,  Kramer,  Vanasse,  &  Donchln,  1983).  In  fact,  in  the 
investigation  by  Wickens,  Kramer,  Vanasse,  and  Donchin,  in  which 
subjects  performed  a  tracking  task  concurrently  with  an  evoked 
potential -eliciting  task  requiring  the  discrimination  of  stimuli,  there 
was  actually  an  advantage  to  the  Intra-modal  condition,  in  which  visual 
tracking  was  time-shared  with  a  visual  probe  task.  In  a  related  line 
of  research  Klapp  (personal  communications)  has  reported  that 
monitoring  of  two  asynchronous  rhythms  is  easier  If  both  are  presented 
to  the  auditory  modality  than  If  one  is  auditory  and  one  is  visual. 

2.  Integral  and  separable  dimensions.  A  series  of  empirical 
studies  and  theoretical  analyses  carried  out  by  Garner  (1970;  Garner  & 
Fefoldy,  1970)  have  drawn  the  distinction  between  pairs  of  dimensions 
that  are  said  to  be  integral  and  those  that  are  separable.  While  the 
clarity  of  this  distinction  has  been  challenged  recently  (e.g..  Dykes, 
1979;  Smith  &  Baron,  1981),  the  original  view  proposed  that  Integral 
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dimensions  were  those  in  which  the  level  along  one  dimension  could  not 
be  specified  without  specifying  the  other  dimension.  As  an  example, 
the  height  and  width  of  a  rectangle  are  said  to  be  integral  because  it 
Is  Impossible  to  specify  the  height  of  a  rectangular  stimulus  without 
also  specifying  its  width  (Dunn,  1983).  If  the  physical  stimulus  did 
not  have  width,  it  would  not  be  a  rectangle. 

In  one  of  Garner's  paradigms  (Garner  &  Fefoldy,  1970),  integral 
dimensions  are  shown  to  provide  efficient  parallel  perceptual 
processing.  This  processing  leads  to  advantages  when  the  dimensions 
are  correlated,  each  calling  for  the  same  response,  but  to 
disadvantages  when  the  dimensions  call  for  different  actions  (see 
Hintzman  et  al . ,  1975). 

Kahneman  and  his  colleagues  have  expanded  upon  this  treatment  by 
arguing  that  the  different  dimensions  of  an  object  (which  generally 
tend  to  be  "Integral"  by  Garner's  criterion),  all  "want"  to  be 
processed  in  parallel  (Kahneman  &  Chajczyk,  1983;  Kahneman  &  Treisman, 
1984).  These  Investigators  provide  data  indicating  the  extent  to  which 
different  dimensions  of  an  object  (color  and  shape)  facilitate  each 
other's  processing  in  a  cooperative  fashion.  An  experiment  by  Lappln 
(1967)  Is  also  highly  consistent  with  these  results.  Lappin  reports 
that  the  color,  size  and  shape  of  a  single  object  can  be  efficiently 
processed  In  a  single  tachistoscopic  exposure  of  the  stimulus.  But 
subjects  have  a  far  more  difficult  time  reporting  the  color  of  one 
object,  the  size  of  another  and  the  shape  of  a  third.  The  "objectness" 
In  the  first  case  fosters  the  parallel  processing  of  its  elements. 
Kahneman 's  recent  research  has  established  that  the  advantages  to 
dimensional  perception  In  a  single  object  are  not  simply  the  result  of 
the  smaller  visual  angle,  and  therefore  better  stimulus  quality 
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relative  to  the  control  conditions  with  which  it  is  compared.  It  is 
the  "objectness"  and  not  the  extent  of  visual  space  that  makes  the 
di fference. 

These  data  are  relevant  to  our  general  discussion  because  they 
suggest  that  the  single  dimensions  of  an  object,  dimensions  which  are 
thereby  spatially  and  psychologically  "closer"  or  more  "proximate"  than 
dimensions  of  separate  objects,  are  better  processed  in  parallel,  and 
therefore  lead  to  more  efficient  time-sharing.  It  is,  of  course,  true 
that  this  proximity  is  not  necessarily  defined  in  terms  of  the  multiple 
resource  space.  For  example,  we  can  assume  that  the  dimensions  of 
either  one  or  three  objects  in  Lappin's  experiment  are  all  processed  by 
the  visual -spatial  resource.  Nevertheless,  this  second  category  of 
results  shares  with  the  first,  the  property  that  time-sharing  is  found 
to  Increase  with  Increased  proximity. 

3.  Manual  control  research.  A  number  of  studies  of  dual  axis 
tracking  have  reached  conclusions  similar  to  those  reported  above: 
that  when  two  tracking  tasks  must  be  carried  out  concurrently, 
performance  of  both  will  be  better  If  they  are  combined  into  a  single 
Integrated  error  indicator  moving  simultaneously  in  the  X  and  Y  axes, 
than  if  they  are  represented  as  two  separate  Indicators  (see  Wickens, 

In  press,  1983  for  a  summary).  In  fact,  Navon,  Gopher,  Chi 1  lag ,  and 
Spitz  (1982)  have  called  attention  to  the  fact  that  time-sharing  can  be 
nearly  perfect  under  such  circumstances.  This  phenomenon  represents  an 
example  of  extremely  good  parallel  processing  of  two  dimensions  of  a 
single  stimulus. 

These  research  findings  in  manual  control  have  been  extended  to 
provide  yet  another  example  of  a  situation  when  time-sharing  efficiency 
Increases  with  Increased  proximity.  This  extention  is  the  observation 
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made  by  Chernikoff,  Duey,  and  Taylor  (1960),  that  dual  axis  tracking 
Mill  be  Improved  when  two  axes  have  the  same  rather  than  different 
dynamics.  This  superiority  will  hold  even  when  the  "different" 
condition  sharing  axes  with  a  higher  and  lower  order  is  compared  with  a 
"same"  condition  in  which  two  tasks  with  higher  order  are  time-shared 
(i.e.,  dual  axis  tracking  is  better  with  two  second  order  control  tasks 
than  it  is  with  a  second  order  task  time-shared  with  one  of  zero 
order).  Presumably  it  is  easier  for  the  subject  to  time-share  two 
independent  axes,  when  only  a  single  "internal  model,"  or  transfer 
function,  need  be  activated  in  working  memory. 

A  subsequent  investigation  by  Chernikoff  and  Lemay  (1963) 
integrated  this  finding  with  the  observation  of  superiority  of  the 
single  tracking  display  reported  by  Navon  et  al.  (1982).  Chernikoff 
and  Lemay  found  that  the  cost  of  having  dissimilar  dynamics  on  two  axes 
was  reduced  If  the  displays  were  also  separate,  but  was  enhanced  if  the 
displays  were  Integrated  Into  a  single  Indicator.  Stated  In  other 
terms,  the  advantage  of  an  Integrated  display  was  reduced  when  the 
dynamics  were  different.  Hence,  these  findings  define  a  sort  of 
"compatibility  of  similarity."  Similar  dynamics  are  most  compatible 
when  they  are  paired  with  similar  (Integrated)  displays. 

What,  If  anything,  is  the  common  theme  that  runs  through  these 
findings,  beyond  the  general  conclusion  that  more  similar  task  or 
similar  display  elements  can^  under  some  circumstances  lead  to  Increased 
time-sharing  efficiency?  A  major  component  appears  to  be  related  to 
the  similarity  or  Integrality  of  task  components.  Thus  In  Garner's 
research,  when  both  task  elements  called  for  a  common  Internal  response 
(I.e.,  there  >  s  redur-*  it  coding  of  two  stimulus  dimensions),  then 
there  was  an  advwtage  to  Integral  (close)  dimensions.  In  the  manual 
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control  research,  when  the  two  axes  called  for  a  common  integrated  mode 
of  internal  processing — that  is,  their  dynamics  were  identical  — 
time-sharing  improved,  and  favored  an  integrated  display  configuration. 

The  theory  that  we  propose  below  asserts  at  its  most  general  level 
that  the  advantage  to  display  proximity  of  two  tasks  will  be  increased 
as  the  correlation/integration  of  task  elements  is  increased.  This 
very  general  statement  needs  elaboration  of  each  of  the  underlined 
terms:  1)  Display  proximity  refers  both  to  similarity  of  display 
elements  within  the  resource  space  and  to  similarity  of  elements  within 
a  given  resource.  Thus,  proximity  may  refer  to  two  visual 
configurations,  as  opposed  to  an  auditory-visual  (bimodal)  one;  or  it 
may  refer  to  two  spatial  configurations,  as  opposed  to  a  spatial  and  a 
verbal  one.  Both  of  these  cases  refer  to  "between-resource" 
similarity.  Alternatively,  spatial  proximity  may  refer  to  two  spatial 
dimensions  of  a  single  object  as  opposed  to  two  dimensions  of  two 
objects  (within-resources  similarity).  2)  The  correlation/integration 
(Cl)  of  task  elements  defines  two  different  concepts:  (a)  External 
correlation  refers  to  the  correlation  of  stimulus  values.  Thus,  the 
pressure  and  temperature  of  a  thermal  process  represent  examples  of 
externally  correlated  variables,  since  high  values  of  one  typically 
accompany  high  values  of  the  other.  So  too  do  the  aileron  position  and 
bank  angle  of  an  aircraft  or.  In  a  different  domain,  the 
years-in-servlce  and  pay-grade  of  Naval  personnel,  (b)  Internal 
correlation  or  integration  refers  to  the  extent  to  which  changes  in  two 
external  stimuli  (whether  these  are  correlated  or  not),  must  be 
combined  together  in  a  single  internal  model,  before  action  can  be 
selected. 

The  air  traffic  control  task  offers  a  good  example  of 
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Integration.  Although  the  flight  courses  of  the  various  aircraft  on  a 
given  display  may  be  uncorrelated  with  each  other  (low  external 
correlation),  the  controller  obviously  needs  to  integrate  the  movement 
of  the  various  aircraft  together  into  a  single  internal  model  of  the 
air  space  before  recommending  an  action  to  any  one  aircraft.  Another 
example  is  provided  by  the  many  attributes  that  must  go  into  a  decision 
of  whether  or  not  a  unit  commander  should  deploy  a  particular  force. 
Here  again  the  attributes  or  cue  values  may  be  unrelated  to  or 
uncorrelated  with  each  other  (e.g.,  readiness  status  of  the  units, 
weather  patterns,  intelligence  reports).  However,  the  decision  clearly 
requires  Integration  of  this  material.  According  to  the  theory  of  task 
integration,  the  air  traffic  control  problem  would  be  a  situation 
ideally  tuned  for  display  proximity,  because  the  integration  is  high. 
Indeed  it  makes  a  great  amount  of  intuitive  sense  to  present  aircraft 
all  within  one  display  format  (vi sual -spatial ) ,  rather  than  to  present 
half  of  the  craft  in  one  format,  and  half  in  the  other--or  to  present 
latitude  spatially  and  longitude  In  terms  of  numerical  coordinates. 

At  the  other  extreme,  consider  a  situation  in  which  integration  is 
not  present.  Consider,  for  example,  control  of  "inner  loop"  flight 
control  variables  and  a  reading  of  weather  patterns  in  an  aircraft. 
Information  for  these  two  tasks  will  best  be  displayed  with  low 
proximity.  There  Is  little  Integration  required  of  these  two  sources 
since  they  essentially  form  two  separate  tasks  rather  than  two  display 
elements  of  a  single  task.  The  two  will  thus  benefit  from  a  display 
using  separate  resources. 

The  general  theory  we  propose  Is  represented  in  Figure  2a.  The 
ordinate  is  some  measure  of  general  task  performance  or  time-sharing 
efficiency  (the  specific  measure  will  be  seen  to  depend  upon  the 
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abscissa  value).  The  abscissa  describes  the  degree  of 
correlation/integration.  This  may  be  zero--in  which  case  the  subject 
is  performing  two  independent  tasks— the  kind  of  time-sharing  research 
most  often  investigated  in  our  laboratory--or  it  may  range  to  near 
“unity."  An  example  of  a  unity  correlation  task  would  be  two  tracking 
tasks  with  identical  input  functions.  An  example  of  a  unity 
integration  task  would  be  one  in  which  two  digits  (the  task  elements) 
must  be  added  to  produce  a  single  output.  Of  course,  correlation  and 
integration  are  two  functionally  independent  concepts.  Hence, 
technically  the  abscissa  should  be  two  axes  and  not  collapsed  into  a 
single  one.  Since  the  theory  predicts  that  both  correlation  and 
integration  will  have  the  same  impact  on  performance  efficiency  and 
proximity,  however,  for  the  moment  we  chose  to  combine  the  dimensions. 
Their  separate  effects  will  be  considered  below. 

The  two  curves  on  the  graph  in  Figure  2a  describe  endpoints  on  a 
continuum  of  "display  proximity."  The  "close"  label  may  refer  to  two 
dimensions  of  a  single  object.  The  "distant"  label  might  refer  to 
information  conveyed  by  the  height  of  a  bargraph  and  the  verbal  content 
of  an  auditory  (spoken)  message.  The  difference  in  slopes  of  the  two 
curves— the  interaction  between  the  two  independent  variables --is  the 
fundamental  property  of  the  proposed  theory. 

We  predict  that  as  task  elements  show  greater 
correlation/integration,  they  will  be  progressively  better  served  by 
display  configurations  that  are  proximate.  The  particular 
representation  in  Figure  2a  shows  a  crossover  interaction.  That  is, 
greater  distance  will  be  better  at  low  levels  of  C/I  and  worse  at  high 
levels.  This  figure  also  shows  a  main  effect  of  C/I  suggesting  that 
tasks  in  general  will  be  easier  if  their  display  elements  are 
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correlated  (this  has  been  demonstrated  in  decision-making  research, 
Phelps  i  Shanteau,  1978;  Medin,  Alton,  Edelson,  &  Freko,  1982)  and/or 
if  they  allow  some  integration  of  their  elements  into  a  single  mental 
model — therefore  creating  more  of  a  single  task  configuration.  It  is 
important  however,  to  realize  that  the  theory  does  not  necessarily 
predict  the  specific  form  shown  in  Figure  2a.  The  forms  shown  in 
Figures  2b,  c,  &  d  are  also  consistent  with  the  basic  tenants,  all  of 
which  show  that  increasing  C/I  will  increase  the  relative  merits  of 
close  versus  distant  configurations,  whatever  the  absolute  merits  of 
each  configuration  may  be. 

As  we  have  noted,  most  of  the  research  in  our  laboratory  has  taken 
place  at  the  far  left  region  of  Figure  2,  and  hence  has  been  consistent 
with  multiple-  resource  theory  predictions  of  superior  performance 
(time-sharing)  efficiency  when  there  is  greater  distance  between 
display  formats.  However  we,  like  Chernlkoff,  Duey,  and  Taylor  (1960) 
have  also  found  that  there  is  improved  performance  with  greater  task 
integration  in  a  manual  control  task,  when  both  tasks  use  the  same 
dynamics  (Wickens  &  Tsang,  1979).  A  recent  investigation  by  Scott  and 
Wickens  (1983)  examined  a  point  closer  to  the  right  of  the  continuum, 
using  an  information  integration/decision  task  in  which  the  reliability 
and  diagnosticity  of  a  sequence  of  information  cues  needed  to  be 
combined  in  order  for  subjects  to  formulate  a  judgment  of  the 
likelihood  that  one  hypothesis  or  the  other  was  in  effect.  We 
concluded  that  performance  when  these  dimensions  were  expressed  as  an 
integrated  object  was  better  than  when  expressed  as  separated  numerical 
values,  a  finding  consistent  with  the  positioning  of  the  right  hand 
part  of  the  two  curves  in  Figures  2a,  c,  and  d.  What  we  have  not  done 
is  to  systematically  vary,  within  an  experiment  the  display  and  task 
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correlation/integration  properties  to  assess  the  nteraction  shown  in 
Figure  2.  An  accompanying  report  (Boles  &  Wlckens,  1983)  partially 
accomplishes  this  purpose  by  determining  if  the  interaction  holds 
across  different  sorts  of  integration  variations,  and  different 
manipulations  of  display  proximity. 

In  more  general  terms,  we  anticipate  that  three  general  applied 
guidelines  could  potentially  flow  from  our  research:  (a)  When  should 
spatial  analog  data  be  mixed  with  verbal  numerical  data,  and  when 
should  data  of  entirely  one  format  be  employed?  (b)  When  should 
"holistic"  object  displays  like  the  iconic  display  of  nuclear  reactor 
safety  parameters  (Wood,  Wise,  &  Hanes,  1981)  be  employed?  (c)  What 
are  the  limits /constraints  on  the  use  of  voice  display?  Voice  display 
in  a  visual  environment  is  an  example  of  "distant"  proximity.  Hence, 
when  the  C/I  is  high,  voice  display  may  not  be  an  effective  means  of 
presenting  information.  The  three  technical  reports  to  follow  will 
address  each  of  these  Issues. 

It  Is  certainly  true  that  there  are  ways  in  which  the  present 
theory  should  be  elaborated.  For  example,  we  have  described  the  C/I 
dimension  as  unitary,  whereas  clearly  stimulus  correlation  and  task 
Integration  may  each  be  varied  Independently  of  each  other.  Thus, 
rightfully  Figure  2  should  be  elaborated  to  the  three  dimensional 
portrayal  of  Figure  3,  thereby  Increasing  the  complexity  somewhat.  In 
this  regard  the  tracking  studies  described  above  provide  data  that 
supports  the  Interaction  of  proximity  with  integration  while  Garner's 
work  on  Integral  and  separable  dimensions  with  correlated  and 
uncorrelated  stimuli  provides  data  that  supports  the  interaction  with 
correlation.  Furthermore,  to  make  matters  still  more  complex,  the 
stimulus  correlation  dimension  may  Itself  be  described  in  terms  of 
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Figure  3:  Predicted  effects  of  correlation  and  integration  on 
performance  efficiency. 


Mickens  6  Boles 


whether  the  correlation  is  0-lag,  or  is  time-lagged  to  varying  degrees 
(e.g.,  the  lag  between  aileron  position  and  bank  angle  in  aviation). 

It  would  appear  that  increasing  time-lag  would  serve  to  reduce  the 
perceived  correlation  and  therefore  shift  a  task  toward  the  low  end  of 
the  C/I  curve. 
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